Remote Direct Memory Access (RDMA) is a technique for efficient movement of data over high-speed transports. RDMA enables a computer to directly place information in another computer's memory with minimal demands on memory bus bandwidth and CPU processing overhead, while preserving memory protection semantics. RNIC is a Network Interface Card that provides RDMA services to the consumer. The RNIC may provide support for RDMA over TCP (transport control protocol).
One of the many important features of the RNIC is that it can serve as an iSCSI (Internet Small Computer System Interface) target or initiator adapter. iSCSI defines the terms initiator and target as follows: “initiator” refers to a SCSI command requester (e.g., host), and “target” refers to a SCSI command responder (e.g., I/O device, such as SCSI drives carrier, tape). The RNIC can also provide iSER (“iSCSI Extensions for RDMA”) services. iSER is an extension of the data transfer model of iSCSI, which enables the iSCSI protocol to take advantage of the direct data placement technology of the RDMA protocol. The iSER data transfer protocol allows iSCSI implementations with the RNIC to have data transfers which achieve true zero copy behavior by eliminating TCP/IP processing overhead, while preserving compatibility with iSCSI infrastructure. iSER uses RDMA wire protocol, and is not transparent to the remote side (target or initiator). It also slightly changes or adapts iSCSI implementation over RDMA; e.g., it eliminates such iSCSI PDUs as DataOut and DataIn, and instead uses RDMA Read and RDMA Write messages. Basically iSER presents iSCSI-like capabilities to the upper layers, but the protocol of data movement and wire protocol is different.
iSCSI protocol exchanges iSCSI Protocol Data Units (PDUs) to execute SCSI commands provided by the SCSI layer. The iSCSI protocol may allow seamless transition from the locally attached SCSI storage to the remotely attached SCSI storage. The iSCSI service may provide a partial offload of iSCSI functionality, and the level of offload may be implementation dependent. In short, iSCSI uses regular TCP connections, whereas iSER implements iSCSI over RDMA. iSER uses RDMA connections and takes advantage of different RDMA capabilities to achieve better recovery capabilities, improve latency and performance. Since RNIC supports both iSCSI and iSER services, it enables SCSI communication with devices that support different levels of iSCSI implementation. Protocol selection (iSCSI vs. iSER) is carried out on the iSCSI login phase.
RDMA uses an operating system programming interface, referred to as “verbs”, to place work requests (WRs) onto a work queue. An example of implementing iSER with work requests is described in U.S. patent application Ser. No. 20040049600 to Boyd et al., assigned to International Business Machines Corporation. In that application, work requests that include an iSCSI command may be received in a network offload engine from a host, and in response to receiving the work request, a memory region associated with the host may be registered in a translation table. As in RDMA, the work request may be received through a send queue, and in response to registering the memory region, a completion queue element may be placed on a completion queue.